Multi-modality medical image viewing

ABSTRACT

A medical imaging system ( 10 ) comprises one or more displays ( 66 ). A viewer device ( 86 ) generates an interactive user interface screen ( 80 ) on the display ( 66 ), which viewer device ( 86 ) enables a user to simultaneously inspect selected image data of multiple patients or multiple images.

BACKGROUND

The present application relates to the medical imaging systems andmethods. It finds particular application in conjunction withmulti-modality systems, such as PET-CT systems. It will be appreciatedthat the invention is also applicable to the various combinations ofSPECT, CT, ultrasound, MRI, fluoroscopy, and the like.

In multi-modality tomographic systems, two or more different sensingmodalities are used to locate or measure different constituents in theobject space. In the PET-CT system, the PET creates images of highmetabolic activity in the body, rather than creating images ofsurrounding anatomy. CT scans allow doctors to see the internalstructures within the human body. Before having a PET-CT scan, thepatient receives a dose of a radiopharmaceutical. The pharmaceutical iscarried through the blood and concentrates in a particular organ orregion and causes radiation to be emitted from the blood and this organor region. During the scan, tracings of the emitted radiation aredetected by the system creating an image of the distribution of theradiopharmaceutical in the patient. The image can show the circulatorysystem and/or the relative absorption of the radiopharmaceutical invarious regions or organs. Integration of the anatomical data from theCT scan with the metabolic data from the PET scan in the PET-CT imagegives physicians visual information to determine if disease is present,the location and extent of disease, and track how rapidly it isspreading. The PET-CT system is particularly helpful indifficult-to-treat regions (e.g. head and neck area, mediastinum,postsurgical abdomen) and localization of the treatment area for thepatients receiving radiation therapy or chemotherapy.

As each medical imaging modality may provide complementary informationon the imaged subject, it is desirable to combine all availableinformation for review. There is a growing demand for a medical imagingreview system to be able to handle multiple patients and multiplemodalities over a temporally spaced series of imaging sessions. However,the current approach for viewing of the multiple patients is to loadpatients one at a time, which is cumbersome from a workflow standardpoint of view and also renders patient comparison difficult if notimpossible.

Another problem arises in handling multiplicity of patients andmodalities. One problem, or example, is the registration of images frommultiple modalities or the same modality over multiple imaging sessions.Current methods allow handling of only few images with the assumptionthat the first volumetric image is fixed. Another problem is inproviding support when conflicting requirements due to different needsexist. One approach is to provide customizable display protocols.However, the customizable display protocols make a tightly integratedviewing environment difficult to define and implement.

The present application provides new and improved methods andapparatuses which overcome the above-referenced problems and others.

SUMMARY

In accordance with one aspect, a medical imaging system which comprisesone or more displays is disclosed. A viewer device generates aninteractive user interface screen on the display, which viewer deviceenables a user to simultaneously inspect selected image data of multiplepatients or multiple images.

In accordance with another aspect, a medical imaging method isdisclosed. An interactive user interface screen is generated on adisplay. Selected image data of multiple patients or multiple images issimultaneously inspected on the display.

One advantage is that multiple images can be simultaneously reviewed ina side to side comparison.

Another advantage is that unlimited support of customer displayprotocols is provided.

Another advantage resides in simplicity of image alignment.

Still further advantages of the present invention will be appreciated tothose of ordinary skill in the art upon reading and understand thefollowing detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may take form in various components and arrangements ofcomponents, and in various steps and arrangements of steps. The drawingsare only for purposes of illustrating the preferred embodiments and arenot to be construed as limiting the invention.

FIG. 1 is a diagrammatic illustration of an imaging system;

FIG. 2 is a diagrammatic illustration of a detailed portion of theimaging system;

FIG. 3 is an image of a user interface screen showing a layout;

FIG. 4 is an image of a user interface screen showing another layout;

FIG. 5 is a diagrammatic illustration of another detailed portion of animaging system;

FIG. 6 is a flow chart of a localization process;

FIG. 7 is a flow chart of a crude registration of comparison groups;

FIG. 8 is a flow chart of precise registration of comparison groups;

FIG. 9 is a flow chart of a registration reformatting which results inoblique reformatting;

FIG. 10 is a flow chart of a layout definition;

FIG. 11 is a flow chart of a layout selection;

FIG. 12 is a flow chart of converting the selecting layout to a viewingnetwork;

FIG. 13 is a continuation of the flow chart of converting the selectedlayout to a viewing network;

FIG. 14 is an image of one of the layouts;

FIG. 15 is an image of a modified layout of FIG. 14;

FIG. 16 is an image showing the layout of FIG. 15 and a newly saved iconfor this layout; and

FIG. 17 is a flow chart of control of oblique TSC display.

DETAILED DESCRIPTION

With reference to FIG. 1, an operation of an imaging multi-modalitysystem 10, which, for example, include any combination of known imagingmodalities, is controlled from an operator workstation 12 which iscoupled to a hospital network 14. The hospital network 14 includesassociated hospital software 16 and a hospital records database 18. Theworkstation 12 may be hardwired to the network 14 or may communicatewith it wirelessly. In one embodiment, the workstation 12 communicateswith other hospital workstations or computers, which are connected tothe hospital network 14, enabling the images and patient records to beforwarded to the appropriate hospital personnel and displayed onassociated monitors. Of course, it is contemplated that imaging system10 is a stand alone system without network connections.

The multi-modality system 10 includes a first imaging system, preferablya functional modality, preferably, a nuclear imaging system 20, and asecond imaging system, such as a computed tomography (CT) scanner 22.The CT scanner 22 includes a non-rotating gantry 24. An x-ray tube 26 ismounted to a rotating gantry 28. A bore 30 defines an examination region32 of the CT scanner 22. An array of radiation detectors 34 is disposedon the rotating gantry 28 to receive radiation from the x-ray tube 26after the x-rays transverse the examination region 32. Alternatively,the array of detectors 34 may be positioned on the non-rotating gantry24. Of course, other imaging systems are also contemplated.

The functional or nuclear imaging system 20, preferably, includes apositron emission tomography (PET) scanner 40 which is mounted on tracks42. Of course, SPECT and other imaging systems are also contemplated.The tracks 42 extend in parallel to a longitudinal axis of a subjectsupport or couch 44, thus enabling the CT scanner 22 and PET scanner 40to form a closed system. A moving means 46, such as a motor and a drive,is provided to move the PET scanner 40 in and out of the closedposition. Detectors 48 are arranged around a bore 50 which defines anexamination region 52. In the PET system, the detectors 48 arepreferably arranged in a stationery ring, although rotatable heads arealso contemplated. In the SPECT system, the detectors 48 are preferablyincorporated into individual heads, which are mounted for rotational andradial movement relative to the patient. A couch moving means 54, suchas a motor and a drive, provides a longitudinal movement and verticaladjustment of the couch 44 in the examination regions 32, 52.

With continuing reference to FIG. 1, an operator, user or imagingtechnician performs a scan using the workstation 12 which includes a CPUprocessor or hardware device 60 and a software component or means 62 forcarrying out the image processing functions and operations. Theworkstation 12 preferably includes one or more input devices 64 (e.g., acomputer keyboard, mouse, etc.), and one or more monitors or displays66. The user, via the input devices 64, can selectively control theworkstation 12 and/or the entire imaging system 10. The couch 44, whichcarries a subject, is moved by the couch moving means 54 into theexamination region 52 for a PET image to be generated by the PET scanner40. Electronic data is reconstructed into a PET image by a PETreconstruction processor 70 and stored in a PET image memory 72. Thecouch moving means 54 moves the couch 44 to position the couch carryingthe subject in the CT scanner examination region 32, where a CT image istaken. More particularly, the couch 44 with the subject is moved to theposition in the CT examination 32 region that is geometrically andmechanically predicted as being the same as its imaged position in thePET imaging region 52. Electronic data is reconstructed into a 3D CTimage by a CT reconstruction processor 74 and stored in a CT imagememory 76.

The PET and CT images are aligned, registered, fused or manipulated inany other appropriate way, after which the image data is appropriatelyformatted by a video processor 78 for display on the monitor 66.

The operator or user controls display of images by using an applicationinterface screen or screens 80 which are incorporated into theworkstation 12 and displayed on the monitor 66. An interface component,processor, or means 82 controls the application interface 80. Theoperator uses the input device, such as a keyboard or mouse 64, tointeract with an applications database 84 by navigating the applicationinterface screens 80.

With continuing reference to FIG. 1 and further reference to FIG. 2, aviewer algorithm, device or means 86 allows the user to view, navigateand compare images of multiple patients. For example, the user selectsmultiple patient data to review and/or compare patients via theinterface screens 80. Selected patient data 88 is passed to the viewerdevice 86 which then creates a patient viewer manager component 90 andpasses the data to it.

The patient viewer manager 90 maintains the selected image data for eachindividual patient. More specifically, the patient viewer manager 90keeps track of which patients are currently being viewed, launches anynew patient for viewing in any order, closes a patient in viewing, andtabs all patients in viewing or tiles all patients in viewing for directside by side comparison. The patient viewer manager 90 can launch apatient by name, or launch a patient before or after a given patient. Ifthe patient is already launched, it is brought up to the front. If thepatient's record is not launched or opened, a new launch is created.

A patient viewing component, algorithm, device or means 92 isresponsible for a given patient viewing. The first patient on the listis chosen for viewing automatically at start-up time. More than twopatients can be displayed in a tiled format which can be in a singledisplay monitor or cross multiple monitors, for direct side by sidecomparison.

The navigations among the patients in viewing are done through clickingon a patient tab or selecting that patient from a graphical userinterface control (a list control). The patient data is loaded into theapplication on-demand to reduce the memory usage. However, it is alsopossible to load all data if the total data does not consume much memoryor if the memory with a proper size is available. In this manner,multiple patients are viewed directly in a single integratedenvironment.

With continuing reference to FIG. 2 and further reference to FIG. 3, alayout manager, component, device, algorithm or means 94 manageslayouts. More specifically, the layout manager 94 maintains a collectionof layouts or layout pool 96. The layouts are obtained from embeddedresources for factory layouts 98 and files for user defined layouts 100.Given a particular set of selected patient image data for reviewing, apredefined startup layout 102 is determined by consulting a preferencesetting. The user may switch to a different layout yet to be created orvisited previously during a reviewing session. The user may modify apredefined layout as shown in FIG. 4 and save the modified layout 104.The user may also start from scratch, define his or her own layout, setit as the preferred one for a particular given set of image data, anduse it in subsequent viewing sessions. For the currently displayedlayout, the user may replace one image data with a different image data.The replacing image data can be displayed in the context of the replaceddata, e.g. in the same layout. Alternatively, the best matching layoutis found and launched. All images are displayed with the same viewingcontext.

With continuing reference to FIG. 2, a registration component, device,algorithm, processor or means 108 selects a global reference image 110and registers or aligns images 112 with one another with respect to theglobal reference image 110. Any number of images can be managed andregistered at a single time. Any image can participate in theregistration as either a reference image or a floating image. Morespecifically, if a single image is selected for viewing, the image isviewed in its own coordinate system. When two or more images areselected for viewing, one image is selected as a reference image and therest of the images is designated as floating image(s). The floatingimages are shifted and rotated to match the reference image. Theindividual relation of each floating image to the reference image iscomputed based on the relation of each image to the global reference asdiscussed in detail below. The initial registration relation isdetermined on the as needed basis. In the registration that involvesmultiple images, the registration relation is propagated to other imagesin a well-controlled manner.

For example, when a group of images (including the case where only oneimage is selected) is selected for viewing for the first time, theregistration device 108 selects the first volumetric image in the groupas the global reference 110 (once the global reference is determined, itwill not change). A global reference coordinate system, which defines acommon frame of reference, is stored in a global reference memory 114.

When an image 112 is selected for viewing, the registration device 108determines and stores a registration matrix (or parameters) for thisparticular volumetric image with respect to the global reference andstores the registration matrix in a registration matrix memory 116.

In one embodiment, the registration device 108 compares the frame ofreference of each image to the common frame of reference, e.g. frame ofreference of the global reference image. If the image has the same frameof reference unique identification (FoR UID) as that of the globalreference and there is no other information to overwrite that, theregistration matrix is identity. If the image and the global referencehave different FoR UID and the registration is not known a priori, theregistration device 108 aligns the volume centers of the two images andpropagates the resultant registration matrix to all other volumetricimages that share the same FoR UID as the image under registration. As aresult, registrations of such images to the global reference image donot need to be determined. If the registration between the image and theglobal reference is known a priori, the registration device 108propagates the determined registration matrix to all other volumetricimages that share the same FoR UID as the image under registration. Theregistrations of such images to the global reference image do not needto be determined. In this manner, registrations of multiple images aresimplified.

The registration of the global reference can be changed. Any change isinterpreted as a change with respect to its original coordinate system.The registration of any image with respect to the global reference isinterpreted with respect to the original coordinate system of the globalreference.

As an alternative, the global reference can be duplicated or copied tobe registered to other images as a floating image. As anotheralternative, the registration matrices of each image can be determinedin a single step rather than on-demand basis.

With continuing reference to FIG. 2 and further reference to FIG. 5, theregistration processor 108 computes a relative registration relationM_(r1r2) between first and second or reference and floating volumetricimages 117, 118 as:

M _(r1r2) =M _(r1) ⁻¹ M _(r2), where

M_(r1) is the registration of the first image with respect to the globalreference; and M_(r2) is the registration of the second image withrespect to the global reference.

The relative registration M_(r1r2) between the reference and floatingvolumetric images 117, 118 is stored in a registration memory 119.

When the relative registration M_(r1r2) of the floating image 118 withrespect to the reference image 117 is changed due to any reasons (manualregistration, automatic registration, etc), the equivalent change withrespect to the global reference image 110 is computed and recorded. Thesame registration is propagated to other images in a controlled manner.

The registration Mr₂ of the floating image 118 with respect to theglobal reference is:

M _(r2) =M _(r1) M _(r1r2).

If the reference image and floating image have different FoR UID, theregistration M_(r2) of the floating image with respect to the globalreference is propagated to all other volumetric images that share thesame FoR UID as the floating image under concern. If the FoR UID is notpresent, it is considered as a mismatch.

If the reference image and floating image have the same FoR UID, tocorrect the motion between CT and PET acquisition, for example, theregistration M_(r2) of the floating image with respect to the globalreference is propagated to all other images that share the same FOR UIDas the floating image under concern and have the same modality as thefloating image.

In this manner, a global reference volumetric image 110 is introducedand all other volumetric images are registered to such virtual globalreference. There is no limit on the number of volumetric images theregistration device 108 can handle and register at a single time. Thecoordinate system of the global reference is not fixed. The globalreference can participate in the registration as a floating image withrespect to other images (the coordinate system of the global referencebefore registration is used as global reference coordinate system andthus is virtual). The registration relation between any two volumetricimages is computed from their respective relation with the globalreference. The individual relation to the global reference can bedecided inside the same application or inferred from outside information(same or different frame of reference unique identification (FoR UID),or DICOM spatial registration object). The global reference isintelligently selected. For example, in the case of mixed volumetric andnon-volumetric images, the global reference is not set until a firstvolumetric image is selected for viewing. Likewise, the individualregistration with respect to the global reference is not determineduntil the image is selected for viewing. If there is no pre-determinedregistration between the global reference and the image under concern,the two centers of the volumetric images are aligned and that alignmentrelation is set as the initial registration that can be overwrittenlater if necessary. Moreover, the so-determined initial registration ispropagated to other volumetric images if their relations to the imageunder concern are known.

When multiple floating images are registered to the same referenceimage, the registrations with respect to the reference image can beupdated as a group with the option that each individual image can beeither updated or not updated.

With continuing reference to FIG. 2, a localization device, algorithm,processor or means 120 uses a maximum intensity projection (MaxIP) ofthe PET image to update other image planes such as transverse, sagittal,and/or coronal images. More specifically, the user selects a point ofinterest, for example, by clicking on a point in the image with themouse. The localization device 120 computes the exact location of theselected point and uses this point's 3D coordinates to update all otherimage planes. The clicked point is computed in the original coordinatesystem and transformed to the common registered coordinate system. Ifthere are more than two images which are registered and displayed, thelocalization device 120 allows the registered and fused images to beupdated properly. Optionally, the MaxIP image, transverse, sagittal,and/or coronal images of other volumetric images can be displayed at thesame time and can be fused with the corresponding transverse, sagittal,or coronal images. As a further option, the MaxIP image can be fused. Inthe case of MaxIP fusion, the user has the option to specify which MaxIPimage from the correlation to be used from this addresses the case inwhich two different physical locations contribute to the same point inthe MaxIP images. In the case that the clicked point is from a regulartwo-dimensional image plane (transverse/sagittal/coronal), a point inthe MaxIP image is also identified where the maximum intensity pixel andthe clicked point are in the same projection line.

More specifically, with particular reference to FIG. 6, the localizationdevice 120 performs a localization process 122 after the user selects apoint on the MaxIP image. In step 122 a, the position of the clickedpoint is retrieved in the screen coordinate system. In step 122 b, thescreen position is transformed into a point in patient coordinate systemby taking into account the image position and image orientation vectors.In step 122 c, the original contributing point in patient coordinatesystem is sought. Optionally, the information can be stored and nofurther computation is needed. In step 122 d, the coordinates found instep 122 c are transformed into a common coordinate system (i.e. of theglobal reference image). In step 122 e, the coordinates in the commoncoordinate system are used to update all associated images. E.g., thetransverse image at the same z position is displayed.

With reference again to FIGS. 3 and 4, a selected point of interest 124is illustrated by a cross. The selected point of interest 124 ispropagated to all associated images to identify corresponding points ofinterest 126.

With reference again to FIG. 5, a comparator 130 compares temporalstudies or groups 132, 134 of selected images. More specifically, theuser selects different groups such as first and second groups of images132, 134 for comparison via the interface screen 80. For example, if twoor more DICOM study objects are selected, the volumetric images in eachstudy form a group. Volumetric images can also be directly selected toform groups. The user can start with one group and use combinations ofkeys and mouse or other devices to select more groups. A two-dimensionalarray is used to represent selected groups, in which each row is agroup. One of the volumetric images, for example a first image in thegroup, is selected as the reference image 117, for example,automatically or manually. The rest of the images of the group aredesignated as floating images 118. All floating images 118 in the samegroup are registered to the reference image 117 in the same group.

Within a group, the floating image has to be informed which image is thereference image; and the reference image has to be informed it is areference image.

After the comparison groups 132, 134 are formed, the comparison groups132, 134 can be displayed independently. A group registration device,algorithm, means or processor 138 determines a linking relation betweengroups that is used in subsequent reviewing. The registration can beestablished in at least two different ways: (1) the crude registrationand (2) the precise registration.

With continuing reference to FIG. 6 and further reference to FIG. 7, thegroup registration processor 138 determines the crude registration in acrude group registration process 140. In step 142, a point correlation(triangulation) is used to identify a common point (hot spot, internalmark, or external mark) in any volumetric image in the first group 132.For example, the user clicks on the point of interest in one of theimages of the first group 132. Step 142 can be performed multiple times.Position of the identified point is stored 144 in a first group memory146. In step 148, the point correlation is similarly used to identifythe same common point in any volumetric image in the second group 134.For example, the user clicks on the similar point in one of the imagesof the second group 134. Step 148 can be performed multiple times.Position of the identified point is stored 150 in a second group memory152. Of course, step 148 can be performed on more than one group.

Alternatively, several common points can be identified in each group132, 134 to improve the registration accuracy. The positions of thepoints are stored in a corresponding first or second group memory 146,152.

After the common points in all groups are identified, the groupregistration processor 138 retrieves the stored positions from the firstand second group memories 146, 152 and the registration is computed 154.The process 140 can be initialized with a clicking on a graphical(button) control. More hue can be provided once the relation iscomputed. Only the relation with respect to the first group is computedregardless of a number of groups selected.

If there is only one point identified in each group, only translationrelation between two groups is considered, which is given by thedifferences between their x/y/z coordinates. If two points areidentified in each group, translation (or translation and one rotation)relation between two groups is considered. If more than three(inclusive) points are identified, the relation can be found by thetechnique used in conjugate-point based registration. It is not requiredto select the same number of points in each group.

With continuing reference to FIG. 6 and further reference to FIG. 8, thegroup registration processor 138 determines 160 the precise registrationbetween groups by using registration between two studies (one volumetricimage from each study group) and linking the groups by this preciserelation. For example, a first image in each group is selected as thereference image. The registration M_(r1) of the reference image 117 ofin the first group 132 with respect to the global reference 110 isretrieved 162. The registration M_(r2) of the reference image of thesecond group 134 with respect to the global reference 110 is retrieved164. The registration M_(r1r2) between the reference image in the firstgroup 132 and the reference image in the second group 134 is determined166 as:

M _(r1r2) =M _(r1) ⁻¹ M _(r2),

where M_(r1) is the registration of the first reference image withrespect to the global reference; and

M_(r2) is the registration of the second reference image with respect tothe global reference.

The registration M_(r1r2) is used as the registration between the twogroups 132, 134. For other groups if any, only the relation with thefirst group 132 is maintained which is determined in the same manner.Alternatively, all pairwise relations can be maintained and used. Oncethe group relation is established, the same relation is used to link thelocalization points in the reference images of each group. Whenever thelocalization point changes in any group, the corresponding point inother groups are computed using the linking relation established earlyand the computed point is set to their groups, which triggers the updatewithin the group.

With reference again to FIG. 2 and further reference to FIG. 9, anoblique processor, device, algorithm or means 168 performs 170 aregistration reformatting to do an oblique reformatting to one of thetwo images. For example, two volumetric images A and B are registeredwith respect to the global reference and the registration matrix 116 isavailable. Alternatively, images A, B share the same FoR UID.Alternatively, images A, B are registered 172, either manually orautomatically, inside or outside the same application and theregistration matrix is available. Alternatively one can directly set theregistration parameters using the provided registration numericalvalues.

In step 174, an oblique image is received. For example, at least one ofthe images A, B is oblique. The image can be oblique when the image isloaded from disk. Alternatively, the image is made oblique reformattedwithin the same application. As a further option, the image itself isnot oblique directly, but the oblique parameters are set and used.

In step 176, an oblique plane is determined. For example, if the obliqueimages are loaded from the disk directly, the oblique plane isdetermined from the image orientation vectors. The cross product ofthose two vectors gives a third vector. The name of the oblique plane(oblique transverse/sagittal/coronal) can be determined by theinformation coded in the image if present; otherwise, it is determinedby comparing the image orientation vectors with respect to the standardx/y/z axes, and if it is closest to the transverse (sagittal or coronal)plane, it is oblique transverse (sagittal or coronal). The comparison isdone on the summation of squared inner products between normalizedvectors and the larger, the closer. In the comparison, the directions ofthose vectors are ignored. Denote the three oblique vectors as u, v, andw, which are closest to the x, y, and z-axes respectively. Generally,the directions of the u, v, and w vectors point to the directions ofpositive x, y, and z-axes respectively; if not, they are negated.

If the oblique images are not loaded from disk files, but generated,then the three oblique vectors can be decided in the same fashion.

A set of geometric information is generated for oblique slice creation,which includes the image orientation vectors, the number of slices, thecenter of the generated slices, the physical extent of the generatedslices, and the number of pixels and pixel physical sizes for theslices. The image orientation vectors are determined by the name of theoblique planes. For example, if an oblique transverse view is requested,two vectors u and v are used. If an oblique sagittal view is requested,y and w are used. If an oblique coronal view is requested, then w and xare used.

In steps 178, 180, the oblique image (Image B) is selected as areference image 117 and an image to be oblique (Image A) is selected asa floating image 118. The floating image 118 is automatically obliquereformatted 182 due to the registration reformatting. Particularly, thegenerated geometric information is used to create oblique slice for theimage A. The geometric information can be used directly. As analternative, the center of the generated slices, the physical extent ofthe generated slices, and the number of pixels and pixel physical sizesfor the slices can vary. When generating the slices for a differentimage, the registration of this image with respect to the referenceimage, on which the geometric information (mainly orientation vectorsand image position) is specified, is taken into account. This process iscalled registration reformatting. User can optionally save the secondaryreformatted images in any oblique plane should it be available.

In this manner, a new workflow concept utilizes the registrationreformatting to do the oblique reformatting, which avoids the manual orautomatic oblique reformatting operation on associated images. In thefusion display, the oblique planes of the oblique image (reference) areused rather than the conventional transverse/sagittal/coronal, i.e. theoblique transverse/sagittal/coronal image planes are used. Due to theregistration reformatting, the other (floating) image is automaticallyoblique reformatted to have the same image planes as the oblique(reference) image. This way, the tedious manual oblique reformatting onthe (floating) image can be avoided or the automatic obliquereformatting becomes unnecessary, which often generates a different setof oblique parameters.

With continuing reference to FIG. 2 and further reference to FIG. 10,the layout manager 94 defines and/or creates a layout definition 196 instep 198, in which layout attributes 200, such as data inputs 202 andviewers 204 are specified 206, and data is associated with the viewers204. The layout definition 196 can be defined in any format of a textfile or a data object as long as it can be easily manipulated. Anexemplary XML format is:

 <name>Axial-comparison-AllF</name> <description>AxialComparison</description>  <writable>false</writable> <favoriteRating>101</favoriteRating>  <inputs>   <Characteristics>   <inputReference>Input1</inputReference>   <modality>Anatomical</modality>   </Characteristics>  <Characteristics>    <inputReference>Input2</inputReference>   <modality>FunctionalAnatomical</modality>   </Characteristics> </inputs>  <ViewerCfgs>   <ViewerCfg>    <width>50</width>   <height>100</height>    <x>0</x>    <y>0</y>   <viewerType>MPR</viewerType>     <fusionState>Underlay</fusionState>   <underlay>     <inputReference>Input1</inputReference>    </underlay>   <overlay>     <inputReference>Input2</inputReference>    </overlay>   <orientation>Transverse</orientation>    <zoomFactor>1.3</zoomFactor>  </ViewerCfg>   <ViewerCfg>    <width>50</width>   <height>100</height>    <x>50</x>    <y>0</y>   <viewerType>MPR</viewerType>     <fusionState>Overlay</fusionState>   <underlay>     <inputReference>Input1</inputReference>    </underlay>   <overlay>     <inputReference>Input2</inputReference>    </overlay>   <orientation>Transverse</orientation>   <enableCine>false</enableCine>    <zoomFactor>1.3</zoomFactor>  </ViewerCfg>  </ViewerCfgs> </Layout>

Such exemplary layout defines two inputs. A first input (“Input1”) looksfor any anatomical volume series. A second input (“Input2”) looks forany supported volume series, which are defined in a modality field. Thevalue of the modality field can also be very specific, for example, MR,CT, PET, SPECT, SC (secondary capture), etc. The exemplary layout nextdefines two viewers (“ViewerCfg”), which have fusion capability. Theviewers show transverse image orientation: one viewer is showingunderlay image and the other viewer is showing overlay image as thestarting viewing option. The layout manager 94 can use this file as aninput, in conjunction with the selected series and a set of built-inlinking rules, to build a complete specification. The layout definition196 is, of course, scalable to include multiple inputs, multiple viewerswith different types, and different viewer configurations. Tables 1-3below explain how to configure each attribute 200 of a layoutdefinition.

TABLE 1 Layout Attribute Type Description writable bool 0: this layoutis read only; 1: layout can be overwritten, and can be removed. namestring Layout name is served as ID. The layout name is unique among alllayouts defined by one physician. The layout name/layout iconname/layout file name are the same. description string Layoutdescription is used as tool tip and item display on preference page.favoriteRating int A score. The higher the value, the better chance thelayout will be selected when multiple layouts match a same query.ViewerCfgs IList A list of all viewer configurations. inputsCharacteritics[ ] Characterized data inputs. Each input forms a query toall selected data from application data model (a software component thatis responsible for packaging and reformatting image data for display).

TABLE 2 ViewerCfg Attribute Type Description Name String Viewer name,which is normally generated by layout manager at runtime if this is notspecified. It is used as a node name by a protocol engine to associatewith a viewer. Width int Width of tab container holding the viewerHeight int Height of tab container holding the viewer X int X locationof the tab container holding the viewer Y int Y location of the tabcontainer holding the viewer screenIndex int 0 - main monitor, 1 -second monitor viewerType ViewerType The exemplary values includeBasic2D MPR MIP CardiacOblique BrainOblique Different viewer type meansdifferent default viewer configuration. For example, for basic 2Dviewing, we do not provide fusion capability for this viewer. For MIP(maximal intensity projection) viewer, we do not provide any ROImeasurement tool. For CardiacOblique viewer and BrainOblique viewer, weprovide some specialized image interactors. underly Characteristics Usedto refer to which input source is used for the underlying data view. Forexample, it is a query among all layout inputs. overlay CharacteristicsUsed to refer to which input source is used for the superimposed dataview. For example, it is as a query among all layout inputs. orientationImageOrientation Values include Any (Basic2D, MIP) Transverse (TSCdisplay) Sagittal (TSC display) Coronal (TSC display) transformedStageTransformedStage Values include NotTransformed (default) Intermediate1Transformed Any values other than NotTransformed are only meaningful foroblique utility layouts. columns unit Rows of tiles in the viewer rowsunit Columns of tiles in the viewer enableCine bool This flag indicatesif cine should be enabled when the viewer is created. zoomFactor floatZoom factor with which the viewer should initially apply to images. Ifit is not specified, then default 1 will be used.viewerConfigurationClass string This attribute allows user to specifyhis/her own viewer configuration class, which is normally done by layoutmanager automatically based on viewer type. extraActions string[ ] Thisattributes allows user to instruct layout manager to execute cannedactions specified here after a viewing network is established.

TABLE 3 Characteristics Attribute Type Description name string Uniquestring used to identify data view generated from this input. It isgenerated internally by protocol builder inputReference InputIndexValues include None Input1 Input2 Input3 Input4 Input5 Input6 — ALL Thisattribute keeps tracking the original input index. With this macro, wecan search all inputs available that meet the criterions presented inCharacteristics data object. modality Modality Values include supportedmodality: Any --- any modality including secondary capture,FunctionalAnatomical - any volume Functional - any functional volumesuch as PET and SPECT, Anatomical - any anatomical volume, PET, CT, MR,SPECT method Method Value include Any (default) Diagnostic (CT) LowDose(CT) CTAC (PET) NonAC (PET) contrastAgent ContrastAgent Enumerationtype. Value include Either (default) NonContrast (not contrast) Contrast(contrast enhanced) groupIndex Int It is used in group comparison. Itindicates which group this input belongs to. Default value is 0 if it isnot specified. ImageType string[] Array of strings requiring all imagetypes in this array to be present in order for an NM image to bematched.

In this manner, users define a balanced layout definition by editing asimple layout file, with which a complete display protocol can be builton the fly based on the current data selection, user preference, andbuilt-in application linking rules. This allows the users to review andinteract with the images in the way that the layout is specified.

With continuing reference to FIGS. 2 and 10 and further reference toFIG. 11, in step 208, the layout device 94 performs data sorting andlayout selection. In step 210, a series characterization is performed.More specifically, some series' properties are retrieved and saved in anobject of characteristics class, which is then put into a hash tablewith the series as its key. Some of other information such as groupindex, which is not part of series' attribute, is obtained from inputgroups.

When the selected data 88 includes multi studies, the layout manager 94performs a study sorting 212 based on studies sorting rules 214 anddecides which study should be displayed first. For example, the studiessorting rules include a rule by which a newer study has a higherpriority than an older study.

In step 220, a series role sorting is performed based on series rolesorting rules 222 to select a series which can serve as a reference. Forexample, a resolution along the z direction is checked in all selectedseries to determine which one in its group is the best for reference. Ifthe resolution fails to tell the difference, then the modality is usedto differentiate them with the following order CT/MR/PET/SPECT.

In step 230, the series are sorted based on series sorting rules 232 todetermine which series will be used first for data matching purpose. Forexample, the series are sorted into two groups such as anatomical andfunctional. For example, the series are sorted based on a modality withfollowing order: PET/SPECT/CT/MR/SC.

For example, functional series, e.g. images generated, for example, withthe nuclear imaging system, are sorted based on following priorities:

Registered over non-registered CT Attenuation Corrected (CTAC) overnon-attenuation corrected (NonAC) Larger field of view (FOV) oversmaller FOV Longer Z coverage over shorter Z coverage (such as wholebody over brain) Static over dynamic Reformatted over non-reformattedNewer over older

For example, anatomical series, e.g. images generated with, for example,the CT scanner, are sorted based on the following priorities:

Higher resolution over lower resolution Larger FOV over smaller FOV(such as low dose CT over diagnose CT) Longer Z coverage over shorter Zcoverage Contrast enhanced over non-contrast enhanced Newer over older

After the sorting, series are picked alternately from functional groupand anatomical group if possible until limit is reached (3 series for anon-comparison case). For example, if one CT, one CTAC PET, one NonACPET and one SC series are selected, then in the final list are CTACPET/CT/NonAC series.

In step 240, all applicable matching layouts 242 among all availablelayouts are identified for data selection. For example, the layoutidentification algorithm is:

Characterize input series for each layout  get all required inputcharacteristics  for each input characteristics   get a list of seriesmatching this characteristics    if there isn't any series in the listwhich is not included     the layout is considered not matched    continue on next layout    else     include the new series and    continue on next required input characteristics  add this layoutinto matched layout list sort all matched layouts

The layouts are selected or sorted 250 based on input matching number,user favorite rating, and layout sorting rules 252 to decide whichlayout should be launched first. For example, a user selects one PET andone CT and there are one layout displaying PET and another displayingPET/CT. The layout sorting rules 252 include a rule by which the PET/CTlayout gets higher priority regardless of favorite rating. Favoriterating starts to play a role when there are more than one layoutmatching the same number of inputs. Another example of the rule is therule which gives a priority to the current layout if the current layoutmatches the current series selection.

In this manner, the optimal layout is automatically selected based onthe data selection. This is based on built-in study and series sortingalgorithms, the list of available layouts and the user favorite rating,and the layout that is currently being displayed.

With continuing reference to FIGS. 2, 10 and 11 and further reference toFIGS. 12 and 13, the layout device 94 converts 260 the selected layoutto a viewing network. The layout manager 94 assigns 262 monitors to theselected layout. For example, some layouts require multiple monitors fordisplay. Effective viewing areas of each monitor are computed and use toconfigure a monitor configure object, which is used by the protocolengine. As a result, the viewers are displayed in monitors where theviewers are supposed to be displayed without crossing a monitorboundary.

In step 264, input macros are expanded. As described above, layoutinputs are defined through characteristics objects, which may includesome macro key words such as “All” for inputReference field. Forexample, all series available that meet the defined criterion, aresearched. The input is expanded into several concrete inputs. Forexample, the layout input contains a macro “All” in the inputReferencefield:

<inputs>  <Characteristics>   <inputReference>All</inputReference>  <modality>PET</modality>  </Characteristics> </inputs>

Assuming there are three series available, among which two series arePET image data and one series is SPECT image data. The input “All” isexpanded into two input definitions, including “Input1” for PET modalityand “Input2” for SPECT modality:

<inputs>  <Characteristics>   <inputReference>Input1</inputReference>  <modality>PET</modality>  </Characteristics>  <Characteristics>  <inputReference>Input2</inputReference>   <modality>SPECT</modality> </Characteristics> </inputs>

After macro expansion, each series are assigned 270 to the layout inputwhich characteristics matches the series.

In step 272, viewer macros are expanded. Generally, every viewer canhave a maximum of two inputs. Each input can potentially have macrossuch as “All” for inputReference. A viewer's input characteristicsdefines a query. If there is more than one series that meet thecriterion for one viewer input, the extra viewer(s) are spawned. Forexample, the extra viewers are tabbed together in the same area. Thescope of the search is limited within all series the layout has beenassigned to.

In step 274, viewer inputs are synched with layout inputs. This stepmakes sure that the viewer's input is assigned with the series.

Once a layout definition is fully populated, a complete display protocol278 is built 280 from the layout definition. The full display protocoldescribes what components are needed and how the components need to beconnected.

In step 282, a node map is registered. Since data views (packed imagedata for viewers to display) are provided by application data model,this step helps to label them with proper names so that they can beconnected properly.

In step 284, data view dimensions are matched for a possible fusionapplication. For example, when one viewer gets assigned two data viewswith different number of dimensions, this step will expand the data viewwith fewer dimensions to match the one with more dimensions. Forexample, a viewer has a CT data as its underlay, which isone-dimensional, e.g. it contains only one volume, and has a PET data asits overlay, which is two-dimensional, e.g. meaning it contains multiplevolumes at the same location but from different times. The CT data isexpanded to have the same number of volumes as the PET data by“duplicating” itself. No actual pixel data is duplicated but only thereference points are added. E.g., the user is allowed to navigatethrough all dimensions of a multiple dimensional image data and to fuseit with other modality data with or without the same dimensions.

In step 288, a viewing protocol is executed. A protocol engine 290 usesthe complete display protocol 278 to instantiate all software objectsand connect them to establish the viewing network.

In step 292, a viewing network is registered with a link controller 294which automatically links viewers. Although the protocol engine 290establishes an initial viewing network, in a fusion display, forexample, color map links need to be changed based on each viewer'sfusion state. The link controller 294 acts as a manager and administersthe entire linking behavior, e.g. adding/removing links. Morespecifically, the link controller 294 automatically links viewers to theassociated image data based on a set of rules 300 including rules for acolor map linking 302, a window level linking 304, triangulation 306 anda zoom/pan presentation linking 308 as described below.

Examples of rules for a color map linking 302:

If same group (in case of comparison)  If common modality   If non-fusedviewer, or same pin if fused viewer    Same color stripe (lookup table)

Example of rules for a window level linking 304:

If same group (in case of comparison)  If common series   Sameupper/lower, window/center

Example of rules for triangulation 306:

If same group (in case of comparison)   Triangulation of all TSC views

Examples of rules for zoom/pan linking 308:

If same group (in case of comparison)  If common orientation(transverse, sagittal or coronal)   Same zoom and pan

In this manner, the viewers are automatically linked to the image dataassociated with the viewers based on a set of built-in linking rules forall layouts. The rules are automatically enforced even when some viewersare destroyed or series is replaced until the user specifically makes achange.

In step 309, the layout manager 94 applies application specific actionsin an orderly manner.

With continuing reference to FIGS. 2 and 10, for easy switching oflayouts, the layout manager 94 preserves 310 the viewing context,including preservation of a color map 312, window level 314, hot spot316, zoom/pan factors 318, image index offset 320, start position 322and free dimension 324. Each viewing context is stored under a certainkey, which is generated based on certain set of rules (in some cases,these are also the rules used to generate across viewer links). Afterlayout is switched, the keys for every viewer and data views attached toit are re-generated. Using the keys, the viewing context is fetched froma common hash table. If such viewing context exists for a given key, itshould be reapplied to the viewer and data view from which the key isgenerated. Of course, it is contemplated that the other features can bepreserved such as region of interest (ROI), labels, tags, markers, andthe like.

To preserve the color map 312, a color map of each data view isretrieved and saved in a hash table under a unique key string which usesnamespace naming convention. The color map will be transferred to nextlayout's data view as long as both layouts share the same group index,the same modality and the same pin (for fused case).

For example, for a fused case, a unique key string is:

Identifier.Modality.Pin.GroupIndex,

where Identifier is “Colormap”;

Modality is one of the following key words “PET”, “CT”, “MR” or “SPECT”;Pin means “underlay” or “overlay” depending on where the data view isattached; and GroupIndex indicates which group the color map is pulledfrom in case of group comparison (for non-comparison layout, the valueis 0).

As another example, for a non-fused case, a unique key string is:

Identifier.Modality.GroupIndex,

where Identifier is “Colormap”;

Modality is one of the following key words “PET”, “CT”, “MR” or “SPECT”;and GroupIndex indicates which group the color map is pulled from incase of group comparison (for non-comparison layout, the value is 0).

The window level is preserved 314 by using a following exemplary keystring:

Identifier.SeriesHashCode.RotationMIP.GroupIndex,

where Identifier is “WindowLevel”;

SeriesHashCode is the hash code of the series; and RotationMIP is aBoolean indicating if the window level is pulled from a MIP viewer(window level is not linked to/from an MIP viewer.)

Since there is only one hot spot per group, the hot spot is preserved316 by using an exemplary key string:

Identifier.GroupIndex,

where Identifier is “Geo”.

Zoom and Pan factors are kept in displayed area window. Only displayedarea window needs to be preserved. Zoom and Pan Factors are preserved318 by using an exemplary key string:

Identifier.SeriesHashCode.Orientation.ViewerType.GroupIndex,

where Identifier is “ZoomPan”;

SeriesHashCode is the hash code of the series; Orientation is imageorientation specified in viewer configuration; and ViewerType is viewertype that is also specified in viewer configuration object.

The image index offset, which is an attribute of viewer's control model,is preserved 320 by using an exemplary key string for a non fused case:

Identifier. SeriesHashCode.Orientation.ViewerType.GroupIndex,

where Identifier is “IndexOffset”;

SeriesHashCode is the hash code of the series; Orientation is imageorientation specified in viewer configuration; and ViewerType is viewertype that is also specified in viewer configuration object.

For a fused case, the image index offset is preserved 320 by usinganother exemplary key string:

Identifier.UnderlaySeriesHashCode.OverlaySeriesHashCode.Orientation.ViewerType.GroupIndex,

where Identifier is “IndexOffset”;

SeriesHashCode is the hash code of the series; Orientation is imageorientation specified in viewer configuration; and ViewerType is viewertype that is also specified in viewer configuration object.

The start position is preserved 322 by using the same key as used topreserve the image index offset. The start position is an attribute ofindexer control model which itself is a part of viewer's control model.

In this manner, viewing context is automatically and intelligentlypreserved when switching layouts. Viewing Context includes color mapsfor modalities of series being reviewed, window levels for series beingreviewed, zoom/pan factor for different image orientations that aredisplayed, lesions that are located by the user, image index, andothers.

Free dimension is preserved 324 by using the same key as used topreserve the image index offset. The free dimension is an attribute ofindexer control model which itself is a part of viewer's control model.

With continuing reference to FIG. 2, a fusion controller 326 controlsdisplay of fused images via a toggle switch 328. Generally, images arefrequently displayed in a non-fused mode with a demand to occasionallyreview images in the fused fashion. In one embodiment, all viewers aredefined as fused viewers, e.g. in the non-fused mode, the viewer statusis changed to a non-fused. The invisible data view is deactivated sothat it will not be affected by any changes occurred in the same vieweror other viewers. If the viewer status is changed to a fused mode, thelink controller 294 first disconnects the viewer and its image data fromits current color map link, and then looks for a suitable color map linkwhich takes care of fused viewers. If such link exists, the linkcontroller includes the new fused viewer along with its data views intothe link. If such link does not exist, the link controller creates sucha viewer. A similar process happens when a fusion is turned off.

Different set of templates, that define what image information is andhow it should be displayed along with image pixel data, is used tohandle image information annotation. When the fusion is turned on, afused annotation template is used. When the fusion is turned off,depending on which data view is visible, a corresponding non-fusedannotation template is used. To avoid confusion, an ROI measurementreflects one image at a time. When the fusion status is changed, an ROIassociation is changed accordingly.

In this manner, the single toggle 328 is provided to turn fusion on/offin any individual viewer within a layout with annotation, region ofinterest measurements and linking behavior automatically adjusted to thechange of the fusion status so that consistent application behavior ismaintained.

With reference to FIGS. 14 and 15, the users are allowed to modifyexisting display and save the modified display as a new layout. Forexample, the user inspects the current display 330 and moves image 332into position 334. All viewers that are on screen are searched. Theirposition, size, fusion state, zoom factors, and monitor information arefetched. Layout inputs are populated when an associated layout isprocessed.

At runtime, layout inputs are populated and they can also be replaced. Afully initialized layout contains no macros. For example, a layout hastwo inputs which are defined as:

<inputs> <Characteristics>   <inputReference>Input1</inputReference>  <modality>Anatomical</modality>  </Characteristics>  <Characteristics>  <inputReference>Input2</inputReference>  <modality>FunctionalAnatomical</modality>  </Characteristics></inputs>

After the layout is fully populated, the inputs of its working copybecome, for example:

<inputs> <Characteristics>   <inputReference>Input1</inputReference>  <modality>MR</modality>  </Characteristics>  <Characteristics>  <inputReference>Input2</inputReference>   <modality>CT</modality> </Characteristics> </inputs>

The same level of modality specificity that the original layout supportsis maintained when the layout is modified and saved. From the exampleabove, when such layout is loaded, only MR & CT can be handled. Toovercome this, the following scheme can be used:

Go through each input  If the populated input supports a subset of datarange that the original input does   Then the original input will beused as the new layout input definition  Otherwise   Promote thepopulated input to the same modality specificity as the original

With reference to FIG. 16, a layout icon 344 is created on the fly. Ascreen capture of an entire image display area is performed. A thumbnailout of the capture is made and saved as a part of the new layout's icon.

With reference to FIG. 17, the users handle the display 350 of theoblique images. In step 352, an oblique angle is detected and measured.When a volume is reformatted, two pieces of oblique information arestored in DICOM file: (1) row/column vectors, and (2) view codesequence, from which the original oblique transformation can be uniquelyconstructed. Once the original oblique transformation is constructed,the original oblique rotation angles or the rotation of each axis fromits original direction can be determined

In step 354, user sets an oblique TSC display state based on the obliqueangle. When a volume is loaded, the user requests permission from theapplication to view the images at the oblique TSC instead of the normalTSC. The permission to use the oblique display is based on rules. Therules, for example, are based on the oblique angle measurement, theexistence of the oblique angle in one of the images, and the layoutconfiguration. For example, the permission is granted if the obliqueangle is from about 0° to about 42.5°. If the volume is not tilted, thepermission is denied. In step 356, the user controls TSC display basedon different oblique angles. The user can set any series as reference,so the actual oblique angle is derived from the reference image. Forexample, a first series has a 10° rotation along axial axis and a secondseries has a 20° rotation along axial axis. The user can display or fusethe first and second series by aligning the first or second series witheither series. The series can be displayed in non-oblique and/or obliquefashion.

In this manner, users can edit a currently displayed layout byre-positioning viewers (components that display images on screen),resizing viewers, changing fusion status of viewers, adjust zoom/panfactor of viewers, destroying viewers, changing viewing series, etc, andcaching it in memory and saving it. This makes creating a viewingnetwork with desired initializations, which used to be the task ofsoftware developers, as easy as saving a modified word processingdocument.

The invention has been described with reference to the preferredembodiments. Modifications and alterations may occur to others uponreading and understanding the preceding detailed description. It isintended that the invention be constructed as including all suchmodifications and alterations insofar as they come within the scope ofthe appended claims or the equivalents thereof.

1. A medical imaging system (10) comprising: one or more displays (66);and a viewer device (86) for generating an interactive user interfacescreen (80) on the display (66), which viewer device (86) enables a userto simultaneously inspect selected image data of multiple patients ormultiple images.
 2. The system as set forth in claim 1, wherein theviewer device (86) includes: a patient viewer manager (90) whichlaunches each patient for viewing in a preselected order; and a patientviewing component (92) which displays image data of each patient.
 3. Thesystem as set forth in claim 2, wherein the patient viewing component(92) displays image data of multiple patients for direct side by sidecomparison on one of a single display (66) or the displays (66).
 4. Thesystem as set forth in claim 1, further including: a layout device (94)which automatically selects a layout for display of the images based atleast on the selected image data and user preferences.
 5. The system asset forth in claim 1, further including: a registration device (108)which identifies a global reference image (110) in the selected imagedata, and determines a registration matrix (116) which aligns an image(112) with the global reference image (110).
 6. The system as set forthin claim 5, further including: a comparator (130) which comparesselected viewing groups (132, 134) of images.
 7. The system as set forthin claim 6, further including: a group registration device (138) whichregisters or aligns images of the selected groups (132, 134).
 8. Thesystem as set forth in claim 7, wherein the group registration device(138) is programmed to perform steps of: identifying at least a firstpoint in a first group image; determining coordinates of the firstpoint; identifying at least a second point in a second group image;determining coordinates of the second point; and computing aregistration relation between the first and second points.
 9. The systemas set forth in claim 7, wherein the group registration device (138) isprogrammed to perform steps of: selecting a reference image in a firstgroup (132); selecting a reference image in a second group (134);computing registration between the images of the first and second groups(132, 134) asM _(r1r2) =M _(r1) ⁻¹ M _(r2), where M_(r1r2) is the registrationbetween the images of the first and second groups, M_(r1) is aregistration of the selected first group reference image with respect tothe global reference image, and M_(r2) is the registration of theselected second group reference image with respect to the globalreference image.
 10. The system as set forth in claim 1, wherein theimages are generated by scanners (22, 40) which include at least one of:PET, SPECT, MRI, ultrasound, fluoroscopy, CT, and digital x-ray.
 11. Amedical imaging system (10) comprising: a memory (114) which storesframe of reference of a selected global reference (110); a registrationdevice (108) which registers images to the global frame of referencesuch that all images are aligned with one another; and a display (66)which concurrently displays aligned images.
 12. The system as set forthin claim 11, further including: a registration matrix memory (116) inwhich the registration device (108) stores a registration matrix whichdefines a relationship between each image (112) and the global referenceimage (110).
 13. The system as set forth in claim 11, wherein theregistration device (108) is programmed to perform steps of: comparing aframe of reference of each image (112) to the global frame of reference;and based on the comparison, one of: determining a registration matrixas an identity matrix; and determining values of the registration matrixbetween the image and the global reference image, and propagating thedetermined registration matrix to the images whose frames of referencecoincide with the frame of reference of the image (112).
 14. The systemas set forth in claim 11, wherein the registration device (108) isprogrammed to perform a step of: determining a registration relationbetween first and second images (117, 118) asM _(r1r2) =M r1 ⁻¹ M _(r2), where M_(r1r2) is the registration betweenthe first and second images, M_(r1) is the registration of the firstimage with respect to the global reference image; and M_(r2) is theregistration of the second image with respect to the global referenceimage.
 15. The system as set forth in claim 14, wherein at least one ofthe images is oblique.
 16. The system as set forth in claim 11, whereinthe images are generated by a multi-modality imaging system whichincludes at least two of: PET, SPECT, MRI, ultrasound, fluoroscopy, CTand digital x-ray.
 17. The system as set forth in claim 11, wherein theimages are generated by imaging different patients.
 18. A medicalimaging system (10) comprising: a localization device (120) whichdetermines a 3D position of a point of interest (124) in one image and,based on the determined position of the point of interest in the image,determines positions of corresponding points of interest (126) inassociated images; and a display (66) which concurrently displays theimages with the determined points of interest (124, 126).
 19. The systemas set forth in claim 18, wherein the localization device (120) isprogrammed to perform steps of: retrieving a position of the point ofinterest (124) in the display coordinate system; transforming thedisplay coordinates of the point of interest (124) into patientcoordinates in accordance with image position and orientation;determining a position of an original point of interest in the patientcoordinate system; transforming coordinates of the original point intothe common coordinate system; and updating the associated images. 20.The system as set forth in claim 18, wherein the localization device(120) propagates image elements identified in the one image to theassociated images, the image elements including at least one of a regionof interest, marker, and label; and wherein the display (66)concurrently displays the images with the image elements.
 21. A medicalimaging system (10) comprising: a memory (114) which stores a frame ofreference of a selected global reference (110); a registration device(108) which registers first and second images (A, B) to the global frameof reference; and an oblique processor or algorithm (168) whichreformats the first image (A) with respect to the second image (B), thesecond image (B) having oblique orientation.
 22. A medical imagingsystem (10) comprising: a display (66); and a layout definition (196)which defines attributes (200) of a layout of the images to be displayedon the display (66) for a view by a user.
 23. The system as set forth inclaim 22, wherein the layout attributes (200) include at least a datainput (202), and a viewer (204).
 24. The system as set forth in claim22, further including: a layout device (94) which builds a completedisplay protocol based at least on the defined layout attributes (200).25. The system as set forth in claim 24, wherein the selected image dataincludes studies including series and the layout device (94) isprogrammed to perform steps of: based on studies sorting rules (214),determining a succession in which the studies are to be displayed; basedon series role sorting rules (222), selecting a reference series in thestudy determined to be displayed first; based on series sorting rules(232) and the selected reference series, determining a preferred orderfor series display; identifying matching layouts based at least on thedetermined preferred order for the series display; based on layoutsorting rules (252) and identified matching layouts, determining asuccession of launching layouts; and executing the determinedpreferential layout.
 26. The system as set forth in claim 25, whereinthe rules (214, 222, 232, 252) include at least one of: a rule by whicha newer study is displayed before an older study, a rule by which aseries with the highest resolution is selected, a rule by which theseries are sorted into at least anatomical and functional groups, a ruleby which the series succession within the group is based on an imagingdevice (22, 40), and a rule by which the layout is selected based atleast on one of a match of the layout attributes (200) and data inputs,and user preference.
 27. The system as set forth in claim 26, whereinthe succession of the imaging devices (22, 40) is PET/SPECT/CT/MR/SC.28. The system as set forth in claim 25, wherein the layout device (94)is further programmed to perform steps of: building the full displayprotocol; and executing the full display protocol.
 29. The system as setforth in claim 28, further including: a protocol engine (290) whichestablishes a viewing network based on the full display protocol. 30.The system as set forth in claim 29, further including: a linkcontroller (294) which dynamically links viewers with an associatedimage data based on linking rules in response to user's interactionssuch as changing fusion states in viewers.
 31. The system as set forthin claim 25, wherein the layout device (94) is further programmed toperform a step of: preserving the viewing context based on a set ofrules which context includes at least one of a color map, a windowlevel, a hot spot, zoom/pan factors, image index offset, start positionand free dimension.
 32. The system as set forth in claim 25, wherein theuser controls the display of oblique images by determining an obliqueangle and setting an oblique display based on the determined obliqueangle.
 33. A medical imaging system (10) comprising: a display (66); anda fusion controller (326) which controls the display of fused and nonfused images on the display (66) with a toggle switch (328).
 34. Thesystem as set forth in claim 33, further including: a link controller(294) which automatically connects or disconnects viewers with anassociated image data based on a status of the toggle switch.
 35. Amedical imaging system (10) comprising: a display (66); and a layoutdefinition (196) which facilitates a user to define attributes (200) ofa layout of images to be displayed on the display (66).
 36. The systemas set forth in claim 35, wherein each layout attribute (200) includes aselectable number of layout elements.
 37. The system as set forth inclaim 35, wherein at least some of the layout elements includemodifiable at least one of a size, location, color, and image type. 38.A medical imaging system (10) comprising: a display (66); a layoutdevice (94) which automatically selects a layout for display of selectedimages based at least on the selected images and user preferences; aninterface screen (80) which facilitates a user to modify the selectedlayout; and a means (82) for saving the modified layout.
 39. The systemas set forth in claim 38, further including: an icon (344) under whichthe modified layout is saved.
 40. A medical imaging method comprising:generating an interactive user interface screen (80) on a display (66);and simultaneously inspecting selected image data of multiple patientsor multiple images on the display.
 41. The method as set forth in claim40, further including: launching each patient in a preselected order;and displaying image data of each patient on the display.
 42. The methodas set forth in claim 41, further including: displaying image data ofmultiple patients for a direct side by side comparison on one or moredisplays.
 43. The method as set forth in claim 40, further including:selecting a layout for display of the images based at least on theselected image data and user preferences.
 44. The method as set forth inclaim 40, further including: identifying a global reference image (110)in the selected image data; determining a registration matrix (116)which aligns a registering image (112) with the global reference image(110); comparing a frame of reference of remaining images to a frame ofreference of the global reference image (110); and propagating theregistration matrix (116) to the images whose frames of referencecoincide with the frame of reference of the registering image (112). 45.The method as set forth in claim 40, further including: identifying aglobal reference image (110); and aligning all images to the selectedglobal reference image (110).
 46. The method as set forth in claim 45,further including: defining a relationship between each image and theglobal reference image (110) via a registration matrix.
 47. The methodas set forth in claim 46, further including: comparing a frame ofreference of each image (112) to the frame of reference of the globalreference image; and based on the comparison, one of: defining anidentity matrix as the registration matrix, and propagating theregistration matrix to the images whose frames of reference coincidewith the frame of reference of the image (112).
 48. The method as setforth in claim 46, further including: determining a registrationrelation between first and second images asM _(r1r2) =M _(r1) ⁻¹ M _(r2), where M_(r1r2) is the registrationrelation between the first and second images, M_(r1) is the registrationof the first image with respect to the global reference image; andM_(r2) is the registration of the second image with respect to theglobal reference image.
 49. The method as set forth in claim 45, furtherincluding: determining a 3D position of a point of interest (124) in oneimage; and based on the determined position of the point of interest inthe image, determining positions of corresponding points of interest(126) in associated images.
 50. The method as set forth in claim 49,further including: retrieving a position of the point of interest (124)in the display coordinate system; transforming the display coordinatesof the point of interest (124) into patient coordinates in accordancewith image position and orientation; determining a position of anoriginal point of interest in the patient coordinate system;transforming coordinates of the original point into the commoncoordinate system; and updating the associated images.
 51. The method asset forth in claim 40, further including: comparing selected viewinggroups (132, 134) of images, the images in each group (132, 134) havinga known registration relation to a common global reference (110). 52.The method as set forth in claim 51, further including: registering oraligning images of the selected groups (132, 134).
 53. The method as setforth in claim 52, further including: identifying a first point in afirst group image; determining coordinates of the first point;identifying a second point in a second group image; determiningcoordinates of the second point; and computing a registration relationbetween the first and second points.
 54. The method as set forth inclaim 52, further including: selecting a reference image in a firstgroup (132); selecting a reference image in a second group (134); andcomputing registration between the images of the first and second groups(132, 134) asM _(r1r2) =M _(r1) ⁻¹ M _(r2), where M_(r1r2) is the registrationbetween the images of the first and second groups, M_(r1) is aregistration of the selected first group reference image with respect tothe global reference image, and M_(r2) is the registration of theselected second group reference image with respect to the globalreference image.
 55. The method as set forth in claim 40, wherein theselected image data includes at least first and second images (A, B),the first and second images (A, B) having a known registration relationto a common global reference (110) and further including: registeringthe first image (A) with respect to the second image (B), the secondimage (B) having oblique orientation with relation to the first image(A).
 56. The method as set forth in claim 40, further including:generating images by at least one of: PET, SPECT, MRI, ultrasound,fluoroscopy, CT, and digital x-ray.
 57. The method as set forth in claim40, further including: defining attributes (200) of a layout of theimages; and displaying the images with the defined attributes on thedisplay (66) for a view by a user.
 58. The method as set forth in claim57, further including: building a complete display protocol based atleast on the defined layout attributes (200).
 59. The method as setforth in claim 58, wherein the selected image data includes studiesincluding series and further including: based on studies sorting rules(214), determining a succession in which the studies are to bedisplayed; based on series role sorting rules (222), selecting areference series in the study determined to be displayed first; based onseries sorting rules (232) and the selected reference series,determining a preferred order for series display; identifying matchinglayouts based at least on the determined preferred order for the seriesdisplay; based on layout sorting rules (252) and identified matchinglayouts, determining a succession of launching layouts; and executingthe determined preferential layout.
 60. The method as set forth in claim59, further including: building the full display protocol; and executingthe full display protocol.
 61. The method as set forth in claim 60,further including: establishing a viewing network based on the fulldisplay protocol.
 62. The method as set forth in claim 61, furtherincluding: automatically linking viewers with an associated image databased on linking rules.
 63. The method as set forth in claim 59, furtherincluding: preserving the viewing context based on a set of rules whichcontext includes at least one of a color map, a window level, a hotspot, zoom/pan factors, image index offset, start position and freedimension.
 64. The method as set forth in claim 59, further including:controlling the display of fused and non fused images with a toggleswitch (328); and automatically connecting or disconnecting viewers withan associated image data based on a status of the toggle switch.
 65. Themethod as set forth in claim 59, further including: modifying theexecuted layout; and saving the modified layout as an icon (344). 66.The method as set forth in claim 59, further including: controllingdisplay of oblique images including: determining an oblique angle; andsetting an oblique display based on the determined oblique angle.
 67. Amedical imaging system for performing the method of claim 40.